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Summary 

Solid State Image Sensors are unlike present day camera tubes in that the light 
sensitive region is organised as discrete light sensitive elements each of which give rise to a 
distinct sample in the output video waveform. 

Because of this spatial sampling structure, and the resultant sampled nature of 
the signal, it has been suggested that there might be an advantage if the sampling fre- 
quency for the sensor were chosen to be the same as that used in any subsequent digital 
processing or signal distribution. 

This report examines the factors affecting the choice of sampling frequencies 
and structures for solid state sensors, and concludes that in general there is no need to 
link the two sampling frequencies, and that, indeed, it will often be undesirable to do so. 
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SAMPLING FREQUENCIES AND STRUCTURES FOR SOLID STATE 

IMAGE SENSING IN BROADCASTING 

G.M. Le Couteur, B.Sc, A.K.C. 



1. Introduction 

It is generally accepted that solid state image sensors 
will eventually replace camera tubes as the means of con- 
verting an image into a television signal in broadcast tele- 
vision cameras. There is also a potential application for 
line sensors as a means of scanning film in telecine machines. 

Predictions as to the likely date of introduction of 
these techniques to broadcast vary somewhat, and this is 
due to the rather uncertain rate of progress of this tech- 
nology in the future, but current informed estimates place 
the date of introduction of solid state sensors into broad- 
casting within the next 5 to 10 years. 

The introduction of digital processing techniques into 
studios and distribution centres is taking place on a some- 
what shorter time scale. One of the questions now receiving 
consideration is the sampling frequency of such digital 
processing and distribution and it has been suggested that 
the most desirable number of elements in a solid state 
sensor may be linked with the choice of an optimum sam- 
pling structure for digital signal processing and that, indeed, 
the two might share a commong sampling frequency. 

Whilst a shared common frequency may seem to be 
desirable on the grounds of simplicity and the avoidance of 
unwanted frequency components related to the frequency 
difference between the two sampling structures, there may 
be irreconcilable differences between the interests of solid 
state sensors (resolution, aliasing, speed of clocking, ortho- 
gonality of sampling structures) and the interests of digital 
signal processing and distribution in which sampling frequen- 
cies are likely to be subcarrier related in PAL and NTSC 
countries and having other structures in SECAM countries. 

This report will particularly examine the sampling 
structures which are possible with solid state sensors, the 
complexity involved in achieving them, and the influence, 
if any, which subsequent digital processing should have on 
the design of sensors. 

2. The sampling structures of solid state image 
sensors 

The sampling structures of solid state sensors are subject 
to certain limitations which arise from the physical pro- 
perties of the devices and the methods which have so far 
been developed to provide internal self-scanning. Certain 
features can be listed which, although not necessarily 
inherent, have, for very good reasons, been adopted in all 
sensors so far built. These are as follows: — 

(a) The arrangement of Solid State Image Sensor targets 
in geometrically ordered light sensitive elements or 
photosites as opposed to homogeneous targets as 
found in most camera tubes. 



(b) The adoption of photosites essentially fixed in space. 

(c) The arrangement of these photosites in an orthogonal 
structure. 

The arrangement of sensor targets in geometrically 
ordered photosites is inherent to the concept of a self- 
scanned sensor. The fact that these should remain per- 
manently fixed in space is highly desirable and may seem 
self evident, although for completeness it should be said 
that the concept of a moveable light sensitive element is not 
as impossible as may appear, since in CCD's, the centres of 
such elements are defined by the position and shape of 
potential wells formed underneath the CCD electrodes, and 
it is possible to alter their shape and position by varying the 
potentials on the electrodes. Slight displacement of 

photosites is carried out to a limited extent in a certain 
type of sensor to achieve interlace,* however large move- 
ment of photosites is no more than a theoretical possibility 



See Appendix. 
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Fig. 2 - Line interlaced sampling structure 

and in practical sensors the need for fixed sampling sites is 
dictated by the internal arrangements for self-scanning. 
Not only does the self-scanning organisation call for fixed 
sampling sites, it also provides strong arguments for wanting 
these sites to be arranged in an orthogonal structure. 

In CCD's two principal methods of internal self- 
scanning have emerged. There are illustrated in Fig. 1(fl), 
ib). Those readers who are unfamiliar with the distinction 
between these two methods will find an explanation in 
Appendix A. Both share the common feature that the 
information is transferred down the sensor along columns 
of parallel transfer registers. This is possible only because 
the light sensitive elements themselves are arranged in rows 
and columns, that is, in an orthogonal structure. 

In present day sensors, therefore, the sampling of the 
optical image is such that the sampling frequencies which 



can be used will either have to be at multiples of line 
frequency, or if a discontinuous sampling sequence is used, 
the frequency can be any desired frequency with the phase 
adjusted at the beginning of each line to make the sample 
phases the same on each line. Any departure from line 
locked or line triggered sampling will produce a video signal 
which will in general contain positional geometric errors, 
either stationary or moving, depending on the precise 
sampling frequency. This obviously rules out the use of 
certain PAL subcarrier-related sampling frequencies unless 
some future extension of the technology yields sensors with 
moveable light sensitive elements. 

Whilst in general the departure from an orthogonal 
sampling structure seems to be undesirable, there are special 
cases of non-orthogonality which might conceivably be 
accommodated by a special design of the sensor. In 
particular, one structure which is being considered for use 
for the sampling of the luminance component in a Y,U,V, 
digital transmission 'package' is the line interlaced pattern 
illustrated in Fig. 2. The sampling pulses are line triggered 
but displaced by 180° on alternate field lines. 

It has been suggested that the possibility of designing 
a solid state sensor with such a sampling structure should 
be considered. Some of the problems facing the designer 
of such a sensor can be understood by reference to Fig. 
3(a) which shows the positions and active areas of the 
photosites on a conventional orthogonal CCD of the frame 
transfer type. (The ensuing arguments will also be 
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Fig. 3 

{a) Positions of photosites in a frame transfer CCD sensor ib) Re-arrangement of photosites to give 'line interlaced' sampling strucutre 

(c) Arrangement of photosites necessary to permit vertical charge transfer 
(rf) High resolution orthogonal structure being used to provide line interlaced sampling by means of interpolation 
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applicable, with some slight accommodation on the part of 
the reader, to sensors of the interline transfer type.) The 
direction of charge transfer into the temporary storage 
region is downwards. 

Fig. 3{b) shows a sensor with the same element 
density, but with a line interlaced structure. it is clear 
that charge transfer channels no longer exist for the transfer 
of charge vertically. One way of overcoming this difficulty 
is to change the shape of the photosites to the form shown 
in Fig. 3(c). Such an arrangement would allow the vertical 
transfer of charge, but would in effect require twice the 
packing density of elements horizontally. If the technology 
permitted packing densities of that order, a preferable 
solution might well be to make an orthogonal structure 
which could then be subsequently signal processed using 
interpolation techniques to give effective apertures of the 
shape shown dotted in Fig. 3{d). The realisation of this 
type of high resolution sensor is, however, unlikely for a 
great many years. 

A further approach which merits possible thought is 
best illustrated by visualising an orthogonal sensor being 
rotated through 45°. The sampling sites will then lie in an 
interlaced pattern as shown in Fig. 4. Charge transfer then 
takes place diagonally. As it stands, this is a poor way of 
using a CCD, because a large part of the available sensitive 
area is not used, the information which emerges is in the 
wrong sequence, and the attainment of interlace seem to 
present rather serious problems. However, it may be that 
the use of CID* technology, which features random access 

* Charge Injection Device. 



direction of 
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Fig. 4 - Frame transfer CCD rotated through 45° to 
demonstrate how a line interlaced structure might be formed 

to the charge on each photosite, could be modified to work 
along these lines, but it is clear that any departure from the 
presently accepted orthogonality of sampling sites will need 
to be justified on very strong theoretical and economic 
grounds, before it could be considered by sensor manu- 
facturers. 
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Fig. 5 

{a) Spectral distribution of scene as modified by the optics of the camera 

{b) Further modification caused by the aperture of the sensor, assumed (sin x)lx 

(c) Formation of alias components by sampling at a frequency/; aliased components shown hatched 

(rf) Characteristics of the filter necessary to avoid further alias components caused by sampling at a second frequency /j 
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3. Using solid state sensors as unsampled, analogue 
picture sources 

Whether or not the structure of solid state sensors can 
be matched to certain line locked sampling structures, it 
seems almost certain that in PAL countries, signal processing 
and/or distribution will take place on non-!ine locked PAL 
subcarrier related frequencies. It is therefore relevant to 
consider the extent to which the output of a solid state 
sensor can be made to look like an unsampled, analogue 
source. There are two aspects to the question which have 
to be considered, namely: 

(a) The effect of optical pre-filtering (or the absence of it) 
on the signal produced by the sensor, and the alias 
components thereby produced. 

(b) The efficiency of removal of frequency components 
consisting of sidebands of the sensor sampling fre- 
quency. This is important because if any such com- 
ponents are left, they will be converted into alias 
components in the second sampling process. 

Fig. 5 illustrates the situation. In Fig. 5(a) the 
spectrum of the optical image is shown. This is likely to be 
approximately Gaussian and as an example it is assumed 
that the required bandwidth is 5-5 MHz, that the energy 
has become negligible by about 11 MHz, and that the 
sampling frequency of the sensor, /is 12 MHz. 

Fig. 5{b) shows the modification to the spectrum 
caused by the aperture of the sensing elements. This 
aperture will be a function of the geometric layout of the 
light sensing elements and of the shape of the potential 
wells beneath them, but the figure illustrates the effect of a 
simple rectangular aperture of width equal to the sample 
pitch; this gives a (sin x)/x spectrum with the first zero at 
the sensor sampling frequency. The overall spectral 
response is given by this (sin x)/x function multiplied by 
the curve in Fig. 5(a). 

Fig. 5(c) illustrates the well-known convolution of the 
spectrum with that of the sensor sampling waveform, and 
shows the consequent overlap between adjacent orders of 
spectrum (alias components) which are shown hatched and 
extend well within the wanted region of the band, i.e. - 
5-5 MHz. 

Fig. 5(c) thus represents the spectral characteristic of 
the signal emerging from the sensor. If it is to be resampled 
at a different frequency (that of the subsequent digital 
processing) it must be low pass filtered to remove the com- 
ponents above the required video bandwidth. 

3.1. Efficiency of remowal of frequency components at 
the sensor sampling frequency 

Assuming that the low pass filter defines the band- 
width of the wanted information by its -3 dB point, a 
criterion can be deduced to define the necessary charac- 
teristics of such a filter. With reference to Fig. 5(d] we see 
that:- 

fs > Vb + A/(_3 to -45) dB 



where /j = sampling frequency of the second sampling 
process (i.e. the digital processing). 

/^ = required bandwidth (usually 5-5 MHz) 

^•''(-3 to -45) dB ~ frequency difference between the 
—3 dB and —45 dB points of the 
filter. 

The figure of —45 dB is tentatively suggested here as a 
reasonable target for the degree of alias suppression neces- 
sary, based upon past experience with the visibility of inter- 
fering signals. 

It should be noted that in this relationship, only the 
resampling frequency f^ is involved, not the sampling fre- 
quency of the sensor. However, this relationship only gives 
the characteristics of a filter designed to remove unwanted 
frequency components arising out of the energy distribution 
in the sampled scene, it does not take into account spurious 
frequencies caused by for example, clock pulse break- 
through occurring in the sensor. 

This breakthrough arises because large voltage levels 
of clock pulse (of the order of 15V) are usually used to 
drive the internal scanning circuits in a sensor. In CCD's 
this breakthrough can occur at the sampling frequency and 
at multiples of it and in some scanning arrangements it can 
even occur at half the sampling frequency. 

The level of breakthrough is always unpredictable, 
but if, for example, it occurred at the sampling frequency/, 
then the second sampling process, at a frequency /j would 
give rise to a beat frequency of /^ - /. Under these circum- 
stances it might well be necessary to supplement the low- 
pass filter response with a notch filter at /and possibly also 
at some of its harmonics. 

It is clear that there are no theoretical obstacles to 
resampling the output at a frequency which is different 
from the sampling frequency of the sensor, the problem is 
purely one of filter design optimised for the particular type 
of sensor and the sampling frequencies used. 

3.2. Alias components caused by the sampling structure 

of the sensor 

A more difficult question to resolve is the tolerable 
level of alias components produced by the sensor itself as a 
result of trying to resolve an optical image containing higher 
resolution than the sensor can reproduce. These are the 
alias components illustrated in Fig. 5(c). 

With a larger number of elements in the sensor, i.e. 
with a higher value of /y, there will be less likelihood of 
serious interference from such components but this is a 
parameter over which there is very little control, since it 
depends on the quality of the camera optics, it being 
virtually impossible to attain an idealised sharp cut spatial 
filter for the optical image. 

For this reason the broadcaster is tempted to ask for 
very high resolution, well in excess of the Nyquist minimum. 
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Fig. 6 

(a) Test Card C as reproduced from film using the 1024 element sensor 

(b) Test Card C as reproduced from film using the 1024 element sensor as a 512 element sensor 
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purely in order to minimise the clanger of encountering 
alias components. The sensor manufacturer, on the other 
hand, will have achieved a great deal if he can even reach 
the Nyquist minimum resolution. 

A compromise between alias components and reso- 
lution will have to be found, but the nature of this com- 
promise will only become clear as a result of subjective 
experience. 

3.3. Experimental results so far 

Whilst experience with high resolution devices is of 
necessity limited at present, line sensors are being used for 
scanning film, using continuous motion of the film to scan 
the image vertically. ' Whilst it is dangerous to generalise 
the results obtained from scanning film to the case of 
television cameras using area arrays, it is nevertheless helpful 
to identify the order of magnitude of the alias problem as 
applied to film. It is in any case felt that telecine machines 
using line sensors may well provide an attractive basis for 
the design of future generations of telecine. 

A 1024 element line sensor is at present under investi- 
gation to assess its suitability for telecine use. One feature 
which makes it useful for investigations into alias formation 
is that the 1024 elements are actually two groups of 512 
elements physically interleaving with each other and having 
distinct outputs. Thus the sensor only becomes a 1024 
element sensor when the two outputs, with their inter- 
leaving sample streams are added together. It is therefore 
possible to view any scene simultaneously as a 512 element 
picture and a 1024 element picture. 

The 1024 (or 512) elements have to be scanned in 
52 ;US, so the corresponding data rates are 19-69 MHz (or 
9-85 MHz), thus the potential Nyquist bandwidths are 
9-85 MHz (or 4-92 MHz). Thus it is possible to compare a 
comfortably oversampled signal of potentially 9-85 MHz 
bandwidth with a system of only just enough elements to 
give a reasonably sharp 4-92 MHz bandwidth picture, but 
with no margin of protection against the formation of alias 
components. 

Fig. 6(fl) and {b) show a test card as viewed under the 
two conditions. There is little obvious difference in reso- 
lution between the two pictures, but the beat frequencies 
on the 512 element image are very obvious in the frequency 
bars. This shows that the frequency bars, being non 
sinusoidal, have a great deal of energy at harmonics extend- 
ing well outside the bandwidth of 4-92 MHz. 

The subjective impression gained when viewing pro- 
gramme material was, however, somewhat different. The 
alias components were seldom visible and never objection- 
able even to engineers well accustomed to seeing effects of 
this type. Although subjective tests were not formally 
carried out it can provisionally be estimated that present 
day 16 mm film is unlikely to contain high frequency com- 
ponents of sufficient energy to be troublesome in a margin- 
ally sampled system. This conclusion is not, however, valid 
for 35 mm film and is, indeed, subject to confirmation by 
future work involving properly conducted subjective tests. 



The above work does not throw a great deal of light 
on the desirable future resolution of solid state sensors in 
television cameras, where the optical image is likely to 
contain far more high frequency energy than that of 16 
mm film. 

The only experimental evidence so far available as to 
the effect of sampling a real scene is the vertical sampling of 
all television images. Alias components produced by this 
process are commonly seen, but have become accepted as 
an unavoidable by-product of the scanning process. By its 
very nature, it is sampled at the Nyquist limit because no 
attempt is made to prefilter the scene or to subsequently 
filter the vertical samples. A conventionally scanned tele- 
vision picture is thus probably a fairly good indication of 
the alias problems likely to be seen on a marginally sampled 
sensor output, but with the alias components then extended 
into two dimensions. 

It should be added that in many of the tests carried 
out on the pictures produced by line sensors, digital pro- 
cessing of the signal has been used. If, for example, it is 
desired to convert the essentially sequentially scanned out- 
put of this type of film scanner into an interlaced sequence, 
digital processing involving storage must be used. On other 
occasions, gamma correction has also been applied digitally. 
At no time has the sampling frequency of the digital pro- 
cessing been linked to that of the sensor; and in every case 
a simple low pass filter has been sufficient to remove 
potentially harmful frequency components from the first 
sampling process. 



4. Conclusions and recommendations 

This report has sought to show that it is both undesir- 
able and unnecessary to attempt to link the sensor sampling 
structure with that of subsequent digital processing. The 
two sampling processes can be independent provided that 
care is taken in the design of the filters necessary to remove 
components arising from the sensor sampling process. A 
simple relationship has been derived to act as a guideline 
in the design of such filters. 

It is felt that the choice of sampling frequency for the 
sensor must above all be dictated by the need to avoid 
serious alias components arising out of the unavoidable lack 
of prefiltering of the optical image, and not by the needs of 
digital distribution and processing, which in the interests of 
cost reduction and bandwidth reduction will tend to use the 
minimum acceptable clock frequencies combined with 
various forms of prefiltering. 

At present it is not possible to make a specific 
recommendation for the optimum number of elements in a 
solid state sensor for use in television cameras owing to the 
lack of experimental evidence, but experience with film 
scanning would suggest that between 500 and 600 hori- 
zontal elements would probably produce visible aliasing, 
comparable with that produced by the existing vertical 
structure of the television system. On the other hand 1000 
elements or so is almost certainly unnecessarily high. It 
therefore seems likely that acceptable results will be 
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achieved by sensors having between 600 and 1000 elements 
in the line direction. 
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Appendix 
Outline description of the two principal methods of scanning Charge Coupled Devices 



1. Interline transfer 

With reference to Fig. 1(«) it can be seen that the 
light sensitive elements are arranged in columns interleaving 
with columns of storage elements. The storage elements 
feed into a horizontal output register. 

Integration of light takes place for the duration of a 
picture. At the end of this period, during the field blanking 
interval, every other light sensitive element in each column 
is made to transfer its charge into the neighbouring storage 
element, the storage elements being shielded from the light. 
(The remaining light sensitive elements contain information 
for the interleaving field and are not used until the subse- 
quent field blanking interval.) 

During the course of the ensuing field, the video 
information in the storage columns is clocked line by line 
into the horizontal output register, and the contents of each 
line are read out of this register during the active line 
period. 

Whilst this arrangement provides a true interlaced 
scanning sequence, the sensitivity is impaired by having half 
the available illuminated area shielded for use as storage. 



2. Frame transfer 

With reference to Fig. 1(Z>), it can be seen that in this 
arrangement the light sensitive areas and storage areas are 
separated into two adjoining areas, with the horizontal out- 
put register below the storage area. 

Both sets of elements, light sensitive and storage 
should be regarded as being in vertical columns, each pair of 
columns comprising a continuous shift register. 

Light integrates in the light sensitive area for a field 
period, and then during the field blanking interval the 
accumulated charge is shifted rapidly down into the shielded 
storage area. During the ensuing field the video in the 
storage area is clocked line by line into the horizontal out- 
put register and read out during each active line. 

Interlace is achieved by varying the centre of the 
light sensitive elements in alternate fields by adjustment of 
potential well shape. Although the interlace thereby 
achieved is fairly satisfactory, the vertical aperture is wider 
than in the corresponding interline transfer case. 
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